It is well known that the dynamic range of an image captured with an image capture device (such as a photographic negative) is often greater than the dynamic range of the output medium (such as a photographic paper or CRT monitor). The result of this incongruity is that a good deal of scene content is rendered to black or white on the output image. For this reason, in an image processing environment, a tone scale function may be used to reduce the scene dynamic range in order to map more information onto the output medium, in a process called dynamic range modification or dynamic range compression. The dynamic range compression modifies the tone scale characteristics of the image.
There exist many processes for creating a tone scale function on an image dependent basis (e.g. see U.S. Pat. No. 5,471,987, issued Dec. 5, 1995 to Nakazawa et al.). Each of the conventional tone scale function processes examines certain statistical characteristics of the image under consideration in order to automatically generate the tone scale function. In addition, the tone scale function may be generated with manual interactive tools by a human operator.
After the tone scale function has been generated, there exists the question of how to apply the tone scale function to the digital image. The goal of dynamic range compression is to adjust the overall dynamic range of the image, rather than to affect the contrast of any given object in the image. In essence, tone scale function should be applied to an image in such a way as to minimize the effect to the scene texture. To that end, it is known to apply the tone scale function to a low frequency sub-band of the image, preserving the higher frequency sub-band(s) that are considered image texture (e.g. see U.S. Pat. No. 5,012,333, issued Apr. 30, 1991 to Lee et al.).
In U.S. Pat. No. 5,012,333, Lee describes a procedure for preserving the high frequency detail of an image by blurring the image neutral channel in order to create a low-pass signal. Subtracting the low-pass signal from the image neutral channel produces a high-pass signal. The processed image is generated by applying the tone scale function to the low-pass signal and adding the result to the high-pass signal. This procedure preserves a segment of the image frequency spectrum, however, artifacts are seen at object boundaries in the image. Gallagher and Gindele build on this work; see U.S. Pat. No. 6,317,521, issued Nov. 13, 2001. More specifically, Gallagher et al. incorporates an artifact avoidance scheme along with a single standard FIR filter to generate the texture signal. While this improvement reduces the occurrence of artifacts in the final image, the artifacts can still be visible.
Also, in U.S. Pat. No. 5,454,044, issued Sep. 26, 1995, Nakajima suggests modifying the image contrast by a formula Sproc=Sorg+f(Sus), where Sproc is the processed image; Sorg is the original image; and f(Sus) is a monotonic function of the unsharp image Sus. In U.S. Pat. No. 5,905,817, issued May 18, 1999, Matama describes using an IIR filter in essentially the same framework as Lee. The advantage of this approach is a reduction in the computational resource requirements.
Each of these methods of applying a tone scale function to an image channel relies on a single blurring with a linear filter. Because of this, there is an inherent size selectivity property in the tone scale function application process. Image structures that are spatially smaller than a certain size are preserved, while details larger than that size are affected by the tone scale function. In addition, the preservation of high frequencies in the image may lead to the creation of unsharp mask type artifacts (overshoot and undershoot) in the neighborhood of large image edges (characteristic of large occlusion boundaries or dark shadows.) In general, it was observed that larger digital filters (used to create the low-pass signal) result in a more pleasing processed image, except for the fact that the artifacts may become more objectionable.
Another approach to the problem of dynamic range modification is to use nonlinear filtering techniques that essentially preserve edges but blur out detail. In U.S. Pat. No. 5,796,870, issued Aug. 18, 1998, Takeo describes a large, rectangular filter, long in the direction along an edge and short in the direction across the edge. This approach reduces the artifacts at edges, but diagonal edges pose a problem. Further, Nakazawa et al. in U.S. Pat. No. 5,471,987, referenced above, describe using an FIR filter whose weights are determined at each pixel location, based on the absolute value of the differences of pixel intensities between two pixels falling under the digital filter. Finally, Gallagher describes (in U.S. Ser. No. 09/457,036, filed Dec. 8, 1999) an adaptive recursive filter having means to adaptively avoid blurring across edge boundaries. These methods are rather time consuming. In addition, it has been found that optimal dynamic range modification cannot be realized by sensing edges at only a single resolution of the image.
Several authors have introduced methods for achieving dynamic range modification of an image by decomposing the image into multiple resolutions. For example, in U.S. Pat. No. 5,467,404, issued Nov. 14, 1995, and U.S. Pat. No. 5,805,721 issued Sep. 8, 1988, Vuylsteke et al. teach a method of decomposing an image into multiple resolutions and using a pre-determined nonlinear amplitude compression function for the high frequency component in each resolution. A deficiency of this method is that the amplitude at each resolution does not adequately identify whether the signal is part of a large amplitude edge or an image texture. A similar invention was disclosed in U.S. Pat. No. 5,717,791, issued Feb. 10, 1998 to Labaere et al., which describes a similar dynamic range compression scheme using wavelet filters to generate the multiple resolutions.
In U.S. Pat. No. 5,907,642, issued May 25, 1999, Ito describes a method of image enhancement based on processing the detail signals of a multiple resolution image representation. Ito describes suppressing the magnitude of detail signals in situations where the next lower detail signal has small magnitude. In U.S. Pat. No. 5,991,457, issued Nov. 23, 1999, Ito et al. describe a method of generating several band pass detail image signals that are modified by application of non-linear functions to modify the dynamic range of the image.
In U.S. Pat. No. 6,285,798 B1, issued Sep. 4, 2001, Lee describes yet another dynamic range compression method using a multiple resolution representation of an image. Lee describes a method of using wavelet filters to create a plurality of coarse signals and detail signals, modifying the detail signals in accordance with contrast gain signals created by detecting the edges of the coarse scale edges, and adding the modified detail signals to the coarse signals to obtain an output image.
In each of these dynamic range compression techniques using multiple image resolutions, the high frequency (or edge or band pass) components of the multiple resolution representation are modified to affect the image dynamic range. However, it is often inconvenient to operate on the high frequency component of the multiple image resolution. In addition, the characteristics of the high frequency signals vary as a function of the level within the multiple image representation of the image. This variability requires a complicated parameter tuning in order to achieve optimal dynamic range compression without producing objectionable artifacts (such as the aforementioned overshoot and undershoot artifact) using a multiple image resolution representation of the image.
Multiresolution, or pyramid methods as a means of representing images as a function of spatial resolution for image processing, has a long history. Burt and Adelson, described a method of representing a digital image by a series of residual images and a base digital image in their journal article “The Laplacian Pyramid as a Compact Image Code” IEEE Transactions on Communications, Vol. Com-31, No. 4, April 1983. However the method taught by Burt et al. was designed for image compression applications and cannot be used for enhancing the tone scale of a digital image.
The prior art methods of dynamic range compression realize that the characteristics of the filter should vary in accordance with the edges of the image. Dynamic range compression is generally applied to only the luminance channel of an image having a luminance channel and one or more chrominance channels. However, because dynamic range compression is generally applied to the luminance channel of a luminance-chrominance image representation, the edges are determined from only the luminance image channel. Thus, the prior art methods sometimes fail when the edges cannot adequately be determined by the luminance channel of an image alone.
In U.S. Pat. No. 5,682,443, issued Oct. 28, 1997, Gouch et al. describe a method of sharpening an image whereby the fringe data is amplified by a value that is determined by color. The method is not used to modify the tone scale characteristics of the image, just the sharpness. In addition, the color information is not used to aid the filtering process when constructing the original fringe data.
There is a need therefore for an improved method of color digital image processing that improves the tone scale of the image and that avoids the problems noted above.